Top 100 LLM Observability Repositories
Ranking
| Ranking | Project Name | Stars | Forks | Language | Open Issues | Description | Last Commit |
|---|---|---|---|---|---|---|---|
| 1 | langfuse | 34,170 | 3,691 | TypeScript | 291 | 🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. 🍊YC W23 | 2026-09-03 |
| 2 | mlflow | 27,801 | 6,250 | Python | 1458 | The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controllin... | 2026-09-04 |
| 3 | Self-Hosting-Guide | 22,714 | 1,156 | Dockerfile | 24 | Self-Hosting Guide. Learn all about locally hosting (on premises & private web servers) and managing software applications by yourself or your organization. Including Cloud, LLMs, WireGuard, Automati... | 2025-06-27 |
| 4 | openobserve | 21,628 | 1,065 | TypeScript | 534 | Open source observability platform for logs, metrics, traces, frontend monitoring, pipelines and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and El... | 2026-09-04 |
| 5 | agents-towards-production | 21,397 | 2,841 | Jupyter Notebook | 0 | End-to-end, code-first tutorials for building production-grade GenAI agents. From prototype to enterprise deployment. | 2026-09-03 |
| 6 | kubesphere | 17,035 | 2,756 | Go | 341 | The container platform tailored for Kubernetes multi-cloud, datacenter, and edge management ⎈ 🖥 ☁️ | 2026-07-15 |
| 7 | RagaAI-Catalyst | 16,161 | 3,570 | Python | 16 | Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced anal... | 2026-02-11 |
| 8 | bisheng | 11,925 | 1,958 | Python | 113 | BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SF... | 2026-09-03 |
| 9 | tensorzero | 11,717 | 965 | Rust | 0 | TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation. | 2026-06-11 |
| 10 | voltagent | 10,540 | 1,119 | TypeScript | 35 | AI Agent Engineering Platform built on an Open Source TypeScript AI Agent Framework | 2026-08-27 |
| 11 | evidently | 7,885 | 909 | Jupyter Notebook | 240 | Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics. | 2026-08-31 |
| 12 | openllmetry | 7,413 | 1,065 | Python | 124 | Open-source observability for your GenAI or LLM application, based on OpenTelemetry | 2026-08-10 |
| 13 | hertzbeat | 7,384 | 1,327 | Java | 277 | An AI-powered next-generation open source real-time observability system. | 2026-09-01 |
| 14 | plano | 7,036 | 487 | Rust | 109 | Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic. | 2026-08-19 |
| 15 | helicone | 6,132 | 664 | TypeScript | 56 | 🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓 | 2026-08-31 |
| 16 | logfire | 4,452 | 281 | Python | 133 | AI observability platform for production LLM and agent systems. | 2026-09-04 |
| 17 | deepflow | 4,254 | 484 | Go | 164 | eBPF Observability - Distributed Tracing and Profiling | 2026-08-31 |
| 18 | claude-devtools | 3,900 | 297 | TypeScript | 27 | The missing DevTools for Claude Code — inspect session logs, tool calls, token usage, subagents, and context window in a visual UI. Free, open source. | 2026-05-13 |
| 19 | langwatch | 3,522 | 362 | TypeScript | 727 | The platform for LLM evaluations and AI agent testing | 2026-09-03 |
| 20 | pezzo | 3,270 | 279 | TypeScript | 42 | 🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more. | 2026-08-21 |
| 21 | holmesgpt | 3,197 | 467 | Python | 127 | SRE Agent - CNCF Sandbox Project | 2026-09-03 |
| 22 | openlit | 2,739 | 367 | TypeScript | 31 | Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, ... | 2026-09-03 |
| 23 | tokencost | 2,007 | 106 | Python | 19 | Easy token price estimates for 400+ LLMs. TokenOps. | 2025-09-05 |
| 24 | future-agi | 1,919 | 577 | Python | 288 | Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0. | 2026-09-04 |
| 25 | failproofai | 1,844 | 436 | MDX | 20 | Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement. 40 built-in policies, a local dashboard, no account required with a generous f... | 2026-09-03 |
| 26 | trpc-agent-go | 1,763 | 303 | Go | 51 | A Go framework for building production agent systems with graph workflows, tools, memory, A2A, AG-UI, MCP, evaluation, and observability. | 2026-09-04 |
| 27 | monoscope | 1,712 | 68 | Haskell | 7 | Monoscope lets you ingest and explore your logs, traces and metrics. We store these in S3 compatible buckets. Query in natural language via LLMs. | 2026-09-04 |
| 28 | trench | 1,662 | 64 | TypeScript | 3 | Trench — Open-Source Analytics Infrastructure. A single production-ready Docker image built on ClickHouse, Kafka, and Node.js for tracking events. Easily build product analytics dashboards, LLM RAGs, ... | 2026-04-06 |
| 29 | llmgateway | 1,600 | 177 | TypeScript | 31 | Route, manage, and analyze your LLM requests across multiple providers with a unified API interface. | 2026-09-03 |
| 30 | superlog | 1,433 | 108 | TypeScript | 12 | Open-source observability tool that uses AI agents to self-heal your software | 2026-09-04 |
| 31 | prompty | 1,257 | 126 | Rust | 5 | Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandab... | 2026-09-03 |
| 32 | langtrace | 1,229 | 127 | TypeScript | 1 | Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorD... | 2025-11-17 |
| 33 | Pensieve | 1,177 | 50 | TypeScript | 0 | Observe how memory shapes an answer. Pensieve is an interactive system for visualizing, interpreting, and managing how Large Language Models (LLMs) “remember” a user. It bridges the gap between model... | 2026-07-28 |
| 34 | GoModel | 1,110 | 92 | Go | 42 | AI gateway / AI control plane / AI proxy written in Go. Unified OpenAI-compatible and Anthropic-compatible API for OpenAI, Anthropic, Gemini, Groq, xAI, Ollama, vLLM and more. A LiteLLM alternative wi... | 2026-09-03 |
| 35 | langkit | 996 | 74 | Jupyter Notebook | 33 | 🔍 LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). 📚 Extracts signals from prompts & responses, ensuring safety & security. 🛡️ Features include text quality, relevance m... | 2024-11-22 |
| 36 | ongrid | 973 | 209 | Go | 47 | An ops AI Agent that understands your infrastructure, finds the root cause, and fixes it — right from Slack, Telegram, Lark or DingTalk. | 2026-09-03 |
| 37 | cv-santiago | 827 | 277 | HTML | 3 | Interactive CV with AI chat integration. Built with React 19, TypeScript, Claude API. Chat with my AI avatar about my experience. | 2026-09-01 |
| 38 | Flawless | 782 | 166 | Python | 3 | AI SRE AgenticOps for Kubernetes and cloud infrastructure. | 2026-08-14 |
| 39 | aperture | 732 | 36 | Go | 50 | Rate limiting, caching, and request prioritization for modern workloads | 2025-12-21 |
| 40 | agentacct | 694 | 76 | Python | 2 | See what your coding agents did and what it cost. Breaks each task down into work steps — tools used, files changed, tests run, time and tokens spent. Local-first dashboard for Claude Code, Codex, Ope... | 2026-09-03 |
| 41 | agentsight | 664 | 99 | C | 17 | lightweight system-level observability for AI Agents | 2026-09-04 |
| 42 | ai-gateway | 626 | 56 | Rust | 7 | The fastest, lightest, and easiest-to-integrate AI gateway on the market. Fully open-sourced. | 2025-11-21 |
| 43 | ai-shortVideo-pipeline | 599 | 106 | Python | 5 | End-to-end AI short-video production pipeline. FastAPI orchestration + Spring Boot gateway with multi-model failover, circuit breaker, metering, and full-stack observability. AI quality gating: prompt... | 2026-09-02 |
| 44 | agent-inspect | 499 | 96 | TypeScript | 17 | Local evidence debugger and trajectory-test toolkit for TypeScript AI agents: inspect causal runs, catch wrong tool paths in CI, and share safe offline evidence. | 2026-09-03 |
| 45 | langfuse-python | 463 | 340 | Python | 6 | 🪢 Langfuse Python SDK - Instrument your LLM app with decorators or low-level SDK and get detailed tracing/observability. Works with any LLM or framework | 2026-09-03 |
| 46 | openllmetry-js | 411 | 75 | TypeScript | 26 | Sister project to OpenLLMetry, but in Typescript. Open-source observability for your LLM application, based on OpenTelemetry | 2026-06-18 |
| 47 | smithers | 407 | 50 | TypeScript | 2 | Agent workflows with full observability and time travel: watch every step live, rewind, fork, replay any run. Claude Code, Codex, Gemini, any model or harness. | 2026-09-04 |
| 48 | clawmetry | 404 | 64 | Python | 115 | See your agent think. Zero-config observability & governance for 26 AI agent runtimes: Claude Code, Cursor, OpenAI Codex, GitHub Copilot, Gemini CLI, Cline, OpenHands, Aider, Goose, OpenClaw, NVIDIA N... | 2026-09-04 |
| 49 | nudgebee | 389 | 263 | Go | 39 | Unified CloudOps platform with AI-SRE, AI-FinOps, AI-K8sOps, and the Agentic Automation Builder without fragmented tools, context switching, or model lock-in. | 2026-09-03 |
| 50 | error-monitoring-agent | 388 | 53 | Python | 0 | Intelligent error monitoring agent that uses Airweave to find context from your code, tickets, and Slack. Clusters similar errors, enriches with relevant context, and creates actionable alerts. | 2026-01-29 |
| 51 | agent-prism | 387 | 23 | TypeScript | 8 | React components for visualizing traces from AI agents | 2026-07-27 |
| 52 | apo | 376 | 54 | Go | 2 | APO is a comprehensive observability platform combining OpenTelemetry with eBPF. Leveraging LLM to enable automated analysis and troubleshooting 🚀. | 2026-03-25 |
| 53 | aigne-framework | 370 | 182 | TypeScript | 11 | The functional, composable, and typescript-first AI Agent framework for real-world LLM Apps. | 2026-01-25 |
| 54 | tokentelemetry | 343 | 53 | Python | 14 | Token telemetry dashboard for AI autonomous and coding agents — tracks tokens, sessions, tool calls & reasoning across Hermes agent, Claude Code, Antigravity CLI, Codex & more. 100% local. | 2026-09-03 |
| 55 | walkerOS | 343 | 23 | TypeScript | 1 | Open-source tag manager for developers | 2026-09-02 |
| 56 | brain0 | 332 | 11 | Rust | 1 | The black box for AI-written code. Passive decision graph linking every commit to the agent prompts behind it: drift detection, DLP audit of what agents read, evidence-driven risk, MCP memory for codi... | 2026-07-19 |
| 57 | llm-bridge | 326 | 33 | TypeScript | 1 | Interoperability between input formats of various LLMs, with observability, error handling, etc. built in. | 2026-05-31 |
| 58 | llmio | 312 | 39 | TypeScript | 15 | Unified LLM gateway with weighted load balancing, observability & cost tracking. 统一的 LLM 网关，提供权重负载均衡、可观测性与费用追踪。 | 2026-08-16 |
| 59 | moneat | 292 | 12 | Kotlin | 21 | 2026-09-03 | |
| 60 | aura | 291 | 23 | Rust | 106 | AURA is a production-tested SRE agent platform you can deploy in minutes. AURA handles the guardrails, APIs, state management, streaming, and failure handling required to put AI to work safely on pro... | 2026-09-03 |
| 61 | agentglass | 289 | 33 | TypeScript | 35 | 🛰 Every AI coding agent on your machine, on one screen — live cost, tokens and tool calls across every provider, and a hold on anything dangerous until you say go. From your desk or your phone. | 2026-09-04 |
| 62 | kitaru | 278 | 17 | Python | 21 | Agent traces you can run, not just read. | 2026-09-03 |
| 63 | langfuse-docs | 239 | 294 | MDX | 17 | 🪢 Langfuse documentation -- Langfuse is the open source LLM Engineering Platform. Observability, evals, prompt management, playground and metrics to debug and improve LLM apps | 2026-09-04 |
| 64 | hub | 225 | 34 | Rust | 4 | High-scale LLM gateway, written in Rust. OpenTelemetry-based observability included | 2026-07-16 |
| 65 | opik-mcp | 217 | 35 | Python | 3 | Model Context Protocol (MCP) server for Opik, the open-source LLM observability and evaluation platform, built by Comet. Read traces, log scores, and manage prompts from Claude Code, Cursor, or VS Cod... | 2026-09-03 |
| 66 | knowledgeops-agent | 190 | 17 | Java | 0 | Production-oriented Spring AI platform prototype for RAG, tool calling, async ingestion, JWT/RBAC, and observability. | 2026-08-31 |
| 67 | zabbix-mcp-server | 190 | 36 | Python | 10 | MCP server for the complete Zabbix API - 237 tools, multi-server, OAuth 2.1 + bearer auth, PDF reports, systemd ready. Works with ChatGPT, Claude, VS Code, Codex, JetBrains and any MCP client. | 2026-08-07 |
| 68 | arconia | 187 | 23 | Java | 17 | Arconia is a framework that acts as an add-on for Spring Boot, designed to enhance modern enterprise applications built with Java. You can add Arconia to an existing Spring Boot application to boost d... | 2026-08-30 |
| 69 | ai-agent-automation | 178 | 99 | TypeScript | 9 | Modular AI agent workflow automation platform with schedulers, tools, and observability. https://vmdeshpande.github.io/ai-automation-platform-website/ | 2026-09-03 |
| 70 | AgenticGoKit | 175 | 37 | Go | 30 | Open-source Agentic AI framework in Go for building, orchestrating, and deploying intelligent agents. LLM-agnostic, event-driven, with multi-agent workflows, MCP tool discovery, and production-grade o... | 2026-08-01 |
| 71 | AgentGuard | 173 | 11 | JavaScript | 2 | Real-time guardrail that shows token spend & kills runaway LLM/agent loops. | 2025-07-31 |
| 72 | duragraph | 169 | 1 | Go | 23 | Self-hosted, enterprise-ready AI workflow orchestration with event sourcing, CQRS, fault tolerance, and observability. Single-binary control plane, Python + Go SDKs, embedded Postgres + NATS for zero-... | 2026-09-03 |
| 73 | example-rag-app | 165 | 24 | TypeScript | 1 | Open-Source RAG app with LLM Observability (Langfuse), support for 100+ providers (LiteLLM), Dockerized, Full Type-checking, 100% Test coverage, and more... | 2026-01-15 |
| 74 | Axon | 159 | 12 | TypeScript | 10 | OpenTelemetry-native LLM observability CLI. Point any OTEL exporter at it and watch your LLM/agent traces in real time. | 2026-08-06 |
| 75 | langfuse-js | 158 | 107 | TypeScript | 11 | 🪢 Langfuse JS/TS SDKs - Instrument your LLM app and get detailed tracing/observability. Works with any LLM or framework | 2026-09-03 |
| 76 | oss-llmops-stack | 150 | 8 | - | 1 | Modular, open source LLMOps stack that separates concerns: LiteLLM unifies LLM APIs, manages routing and cost controls, and ensures high-availability, while Langfuse focuses on detailed observability,... | 2026-07-28 |
| 77 | llemonstack | 147 | 26 | TypeScript | 1 | All-in-one local low-code AI agent development platform. Installs and runs n8n, Flowise, Browser-Use, Qdrant, Ollama, and more. Proxies LLM requests through LiteLLM with Langfuse for observability. | 2026-02-03 |
| 78 | multi-agent-ralph-loop | 146 | 21 | Shell | 0 | Autonomous orchestration framework for Claude Code with MemPalace-inspired memory (4-layer stack, 818-token wake-up), parallel-first Agent Teams (6 teammates), Aristotle First Principles methodology, ... | 2026-09-03 |
| 79 | AgentLoom | 145 | 15 | Python | 4 | Simple, flexible workflow orchestration for multi-agent AI apps, with YAML configuration, runtime safety, observability, and resume support. | 2026-08-13 |
| 80 | gopher-mcp | 145 | 19 | C++ | 79 | C++ MCP SDK - build Model Context Protocol (MCP) servers and clients in C++ / CPP. Enterprise-grade security, observability, connectivity. Stdio, HTTP+SSE, Streamable HTTP, WebSocket, TCP transports. ... | 2026-09-02 |
| 81 | NeMo-Relay | 141 | 64 | Rust | 3 | Multi-language agent runtime and library for execution scope management, lifecycle events, and middleware on tool and LLM calls. | 2026-09-04 |
| 82 | inferock-bench | 140 | 49 | TypeScript | 0 | Local LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity receipts. | 2026-08-17 |
| 83 | steer | 133 | 5 | Python | 0 | The Active Reliability Layer for AI Agents. Catch failures, teach fixes, and automate reliability | 2026-01-19 |
| 84 | rocketplaneIO | 131 | 4 | Go | 0 | Self-hosted AI SRE for Kubernetes — zero-instrumentation eBPF observability plus a copilot that fixes issues through guardrailed, self-verifying actions. BYO-LLM, air-gapped capable. | 2026-07-29 |
| 85 | aisix | 129 | 26 | Rust | 105 | Open-source AI gateway for LLMs & AI agents, built in Rust. One OpenAI-compatible API for OpenAI, Anthropic, Gemini, Bedrock & more — routing, guardrails, caching, rate limits, observability. | 2026-09-04 |
| 86 | language-ai-engineering-lab | 127 | 27 | Jupyter Notebook | 0 | Language AI Engineering Lab, a place where you can deeply understand and build modern Language AI systems, from fundamentals to production. | 2026-06-24 |
| 87 | ai-runtime-lab | 126 | 4 | Python | 6 | Engineering deterministic, production-grade systems around non-deterministic LLMs — FSM, durable execution, retries, DAGs, agent runtimes, model routing, edge inference, RAG, memory, multi-agent orche... | 2026-07-20 |
| 88 | agentwatch | 124 | 11 | Python | 0 | A powerful AI observability framework that provides comprehensive insights into agent interactions across platforms, enabling developers to monitor, analyze, and optimize AI-driven applications with m... | 2025-05-14 |
| 89 | agentburn | 122 | 2 | Python | 2 | Which 5-hour window took you out — and where the money goes. Local profiler for Claude Code, OpenClaw & Hermes Agent: usage windows, burn by source (cron/chats/subagents), the overnight bill, why it b... | 2026-08-31 |
| 90 | agentmaker | 121 | 4 | Python | 0 | A general-purpose Python framework for building LLM agents and multi-agent systems. "Four lines of code, an agent with memory." | 2026-08-20 |
| 91 | generative-ai-project-template | 118 | 11 | Python | 1 | Template for a new generative ai project using uv, nicegui, fastapi, llms (cloud & local with litellm and ollama, cpu/gpu) and langfuse for llm observability | 2026-09-03 |
| 92 | tma1 | 117 | 14 | Go | 1 | Local-first observability your agent reads back. TMA1 records every LLM call, then routes what it sees into the agent's next turn via hooks and MCP. | 2026-09-03 |
| 93 | agenthub | 111 | 12 | Python | 1 | One interface for 1,000+ LLMs, with zero-code switching and built-in observability. (GPT-5.6 / Claude 5 / Gemini 3.7) | 2026-09-03 |
| 94 | AgC | 111 | 9 | Kotlin | 21 | AgC is the open-core platform that powers Open Agentic Compute — a new compute substrate purpose-built for deploying, running, and orchestrating AI agents at scale. | 2026-07-10 |
| 95 | AIephant-AI-Agent-Gateway | 110 | 15 | Rust | 0 | Alephant is an open-source AI Agent Gateway for routing, tracking, and controlling LLM usage across AI agents, members, and workflows, and for publishing agent capabilities as paid endpoints with x402... | 2026-06-12 |
| 96 | traccia-py | 109 | 20 | Python | 0 | OpenTelemetry-native SDK for AI agent observability, tracing, evaluation, debugging, governance, and runtime policy enforcement. Framework-agnostic and built for OpenAI Agents, LangGraph, CrewAI, and ... | 2026-08-24 |
| 97 | Fastllm-proxy | 107 | 97 | Rust | 2 | The lowest-overhead LLM router. Production-ready, highly available, one OpenAI-compatible endpoint in front of 80 providers and your own vLLM/SGLang — 0.76 µs per request, no I/O on the request path, ... | 2026-08-20 |
| 98 | spark-dashboard | 106 | 26 | TypeScript | 4 | Real-time hardware and LLM inference monitoring — GPU, CPU, memory, and vLLM metrics streamed to a dashboard. | 2026-09-03 |
| 99 | tokenjam | 106 | 28 | Python | 53 | Token Efficiency For AI Agents | 2026-09-03 |
| 100 | JDOxyGent4J | 105 | 19 | Java | 1 | [ACL 2026] OxyGent: Making Multi-Agent Systems Modular, Observable, and Evolvable via Oxy Abstraction https://arxiv.org/abs/2604.25602 | 2026-06-30 |